Method and system for robot manipulation planning

ABSTRACT

A method for planning a manipulation task of an agent, particularly a robot. The method includes: learning a number of manipulation skills wherein a symbolic abstraction of the respective manipulation skill is generated; determining a concatenated sequence of manipulation skills selected from the number of learned manipulation skills based on their symbolic abstraction so that a given goal specification indicating a given complex manipulation task is satisfied; and executing the sequence of manipulation skills.

CROSS REFERENCE

The present application claims the benefit under 35 U.S.C. § 119 of European Patent Application No. EP 19181874.9 filed on Jun. 21, 2019, which is expressly incorporated herein by reference in its entirety.

FIELD

The present invention relates to automatic processes for planning tasks by determining a sequence of manipulation skills. Particularly, the present invention relates to a motion planning framework for robot manipulation.

BACKGROUND INFORMATION

General use of robots for performing various tasks is challenging, as it is almost impossible to preprogram all robot capabilities that may potentially be required in the latter application. Training the skill whenever it is needed renders the use of robots inconvenient and will not be accepted by a user. Further, simply recording and replaying a demonstrated manipulation is often insufficient, because changes in the environment, such as varying robot and/or object poses, would render any attempt unsuccessful.

Therefore, the robot needs to recognize and encode the intentions behind these demonstrations and should be capable to generalize the trained manipulation to unforeseen situations. Furthermore, several skills need to be performed in a sequence to accomplish complex tasks. The task planning problem aims to define the right sequence of actions and needs a prespecified definition of the planning model and of the preconditions and effects of all available skills. Due to the large variation of skills, the definition of such a planning model quickly becomes impractical.

SUMMARY

According to the present invention, a method for planning an object manipulation, and a system for planning object manipulation are provided.

Further embodiments are described herein.

According to a first aspect of the present invention, an example method for planning a manipulation task of an agent, particularly a robot, is provided, comprising the steps of:

-   -   Learning (Training) a number of manipulation skills (a_(h))         wherein a symbolic abstraction of the respective manipulation         skill is generated;     -   Determining a concatenated sequence of manipulation skills         selected from the number of learned manipulation skills based on         their symbolic abstraction so that a given goal specification         indicating a given complex manipulation task is satisfied;     -   Executing the sequence of manipulation skills.

Further, the learning of the number of manipulation skills may be performed in that a plurality of manipulation trajectories for each respective manipulation skill is recorded, particularly by demonstration, a task parametrized Hidden Semi-Markov model (TP-HSMM) is determined depending on the plurality of manipulation trajectories for each respective manipulation skill and the symbolic abstraction of the respective manipulation skill is generated.

The above task planning framework allows a high-level planning of a task by sequencing general manipulation skills. Manipulation skills are action skills in general which may also include translations or movements. The general manipulation skills are object-oriented and respectively relate to a single action performed on the object, such as a grasping skill, a dropping skill, a moving skill or the like. These manipulations skills may have different instances, which means that the skills can be carried out in different ways (instances) according to what is needed to be done next. Furthermore, the general skills are provided with object centric symbolic action descriptions for the logic-based planning.

The above method is based on learning from demonstration by means of fitting a prescribed skilled model, such as a Gaussian mixture model to a handful of demonstrations. Generally, a TP-GMM task parameterized Gaussian mixture model may be described which can then be used to an execution to reproduce a trajectory for the learned manipulation skill. The TP-GMM is defined by one or more specific frames (coordinate systems) which indicates the translation and rotation with respect to a word frame. After observation of the actual frame, the learned TP-GMM can be converted into a single GMM. One advantage of the TP-GMM is that the resulting GMM can be updated in real time according to the observed task parameters. Hence, the TP-HSMM allows to adapt to changes in the objects during the execution of the manipulation task.

Furthermore, the generating of the symbolic abstraction of the manipulations skills may comprise constructing a PDDL model, wherein objects, initial state and goal specification define a problem instance, while predicates and actions define a domain of a given manipulation, wherein particularly the symbolic abstraction of the manipulations skills uses a classical PDDL planning language.

It may be provided that the concatenated sequence of manipulation skills is determined, such that the probability of achieving the given goal specification is maximized, wherein particularly a PDDL planning step is used to find a sequence of actions to fulfill the given goal specification, starting from a given initial state.

According to an example embodiment of the present invention, the transition probability between states of the TP-HSMM may be determined using Expectation-Maximization.

Moreover, a task parametrized Hidden Semi-Markov model (TP-HSMM) may be determined by cascading manipulations skills, wherein a Viterbi algorithm is used to retrieve the sequence of states from the single TP-HSMM based on the determined concatenated sequence of manipulation skills.

Parameters of the TP-HSMM may be learned through a classical Expectation-Maximization algorithm.

Furthermore, the symbolic abstractions of the demonstrated manipulation skills may be determined by mapping low-variance geometric relations of segments of manipulation trajectories into the set of predicates.

According to an example embodiment of the present invention, the step of determining the concatenated sequence of manipulation skills may comprise an optimization process, particularly with the goal of minimizing the total length of the trajectory.

Particularly, determining the concatenated sequence of manipulation skills may comprise selectively reproducing one or more of the manipulation skills of a given sequence of manipulation skills so as to maximize the probability of satisfying the given goal specification.

Furthermore, determining the concatenated sequence of manipulation skills may include the steps of:

-   -   Cascading the TP-HSMMs of consecutive manipulation skills into         one complete model by computing transition probabilities         according to a divergence of emission probabilities between end         states and initial states of different manipulations skills;     -   Searching the most-likely complete state sequence between the         initial and goal states of the manipulation task using a         modified Viterbi algorithm.

Particularly, the modified Viterbi algorithm may include missing observations and duration probabilities.

According to a further embodiment of the present invention, a device for planning a manipulation task of an agent, particularly a robot, is provided, wherein the device is configured to:

-   -   learn a number of manipulation skills, wherein a symbolic         abstraction of the respective manipulation skill is generated;     -   determine a concatenated sequence of manipulation skills         selected from the number of learned manipulation skills based on         their symbolic abstraction so that a given goal specification         indicating a complex manipulation task is satisfied; and     -   instruct execution of the sequence of manipulation skills.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments of the present invention are described in more detail in conjunction with the figures.

FIG. 1 schematically shows a robot arm.

FIG. 2 shows a flowchart illustrating the method or manipulation of an object by sequencing of manipulation skills.

FIG. 3 shows illustrations for HSMM states of a learned manipulation skill wherein demonstration trajectories and transition probabilities are illustrated.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 shows a system (agent) 1 including a controllable robot arm 2 as an example of an object manipulator. The robot arm 2 is a multi-DoF robotic arm with several links 21 and an end-effector 22 which allows a state x∈

³×S³×

¹ (describing the Cartesian position, orientation and gripper state in a global coordinate system (frame)), that operates within a static or dynamic and known workspace. Also, within the reach of the robot arm 2, there are objects of interest denoted by O={o₁, o₁, . . . , o_(J)}.

Within this setup, a human user can perform several kinesthetic demonstrations on the arm to manipulate one or several objects for certain manipulation skills. Denote by A={a₁, a₁, . . . , a_(H)} the set of demonstrated skills. Moreover, for manipulation skill a_(h)∈A, the set of objects involved is given by O_(a) _(h) and the set of available demonstrations is denoted by D_(a) _(h) .

The robot arm 2 is controlled by means of a control unit 3 which may actuate actuators to move the robot arm 2 and activate the end effector 22. Sensors may be provided at the robot arm 2 or at the robot workspace to record the state of objects in the robot workspace. Furthermore, the control unit 3 is configured to record movements made with the robot arm 2 and to obtain information about objects in the workspace from the sensors and further to perform a task planning process as described below. The control unit 3 has a processing unit where the algorithm as described below is implemented in hardware and/or software.

All demonstrations are described by the structure of TP-GMM (task-parametrized-Gaussian Mixture Models). The basic idea is to fit a prescribed skill model such as GMMs to multiple demonstrations. GMMs are described, e.g., in S. Niekum et al. “Learning grounded finite-state representations from unstructured demonstrations”, The International Journal of Robotics Research, 34(2), pages 131-157, 2015. For a number M of given demonstrations (trajectory measurement results), each of which contains T_(m) data points for a dataset, N=M*T_(m) total observations ξ={ξ_(t)}_(t=1) ^(N) exists, where ξ_(t)∈

^(d) for sake of clarity. Also, it is assumed the same demonstrations are recorded from the perspective of P different coordinate systems (given by the task parameters such as objects of interest). One common way to obtain such data is to transform the demonstrations from global frame (global coordinate system) to frame p by ξ_(t) ^((p))=A_(t) ^((p)) ⁻¹ (ξ_(t)−b_(t) ^((p))), where {(b_(t) ^((p)),A_(t) ^((p))}_(p=1) ^(P) is the translation and rotation of frame p with respect to the given global frame at time t. Then, a TP-GMM (task-parametrized-Gaussian Mixture Model) is described by the parameters {π_(k),{μ_(k) ^((p)),Σ_(k) ^((p))}_(p=1) ^(P)}_(k=1) ^(K) where K represents the number of Gaussian components in the mixture model, π_(k) is the prior probability of each component, and {μ_(k) ^((p)),Σ_(k) ^((p))}_(p=1) ^(P) are the parameters of the k-th component within frame p. K may be manually assigned. However, there are approaches to automatically set K.

Differently from standard GMM learning, the mixture model above cannot be learned independently for each frame p. Indeed, the mixing coefficients π_(k) are shared by all frames p and the k-th component in frame p must map to the corresponding k-th component in the global frame. For example, Expectation-Maximization (EM) is a well-established method to learn such models. In general, an expectation-maximization (EM) algorithm is an iterative method to find maximum likelihood or maximum a posteriori (MAP) estimates of parameters in a statistical model, which depends on unobserved latent variables.

Once learned, the TP-GMM can be used during execution to reproduce a trajectory for the learned skill. Namely, given the observed frames {b_(t) ^((p)),A_(t) ^((p))}_(p=1) ^(P), the learned TP-GMM is converted into one single GMM with parameters {π_(k),μ_(t) ^((p)),Σ_(t) ^((p))}_(k=1) ^(K), by multiplying the affine-transformed Gaussian components across different frames, as follows

${\left( \hat{\sum\limits_{t,k}} \right)^{- 1} = {\sum\limits_{p = 1}^{P}\left( {\hat{\sum}}_{t,k}^{(p)} \right)^{- 1}}},{{\hat{\mu}}_{t,k} = {\hat{\sum\limits_{t,k}}{\sum\limits_{p = 1}^{P}{\left( {\hat{\sum}}_{t,k}^{(p)} \right)^{- 1}{\hat{\mu}}_{t,k}^{(p)}}}}}$ where the parameters of the updated Gaussian at each frame p are computed as {circumflex over (μ)}_(t,k) ^((p)) =A _(t) ^((p))μ_(k) ^((p)) +b _(t) ^((p)) and {circumflex over (Σ)}_(t,k) ^((p)) =A _(t) ^((p))μ_(k) ^((p)) +b _(t) ^((p))

Hidden semi-Markov Models (HSMMs) have been successfully applied, in combination with TP-GMMs, for robot skill encoding to learn spatio-temporal features of the demonstrations, such as manipulation trajectories of a robot or trajectories of a movable agent.

Hidden semi-Markov Models (HSMMs) extend standard hidden Markov Models (HMMs) by embedding temporal information of the underlying stochastic process. That is, while in HMM the underlying hidden process is assumed to be Markov, i.e., the probability of transitioning to the next state depends only on the current state, in HSMM the state process is assumed semi-Markov. This means that a transition to the next state depends on the current state as well as on the elapsed time since the state was entered.

More specifically, a task parametrized HSMM model consists of the following parameters M _(θ) ={{a _(kh)}_(h=1) ^(K),(μ_(k) ^(D),σ_(k) ^(D)),π_(k),{μ_(k) ^((p)),Σ_(k) ^((p))}_(p=1) ^(P)}_(k=1) ^(K) where a_(kh) is the transition probability from state k to h; (μ_(k) ^(D),σ_(k) ^(D)) describes the Gaussian distributions for the duration of state k, i.e., the probability of staying in state k for a certain number of consecutive steps; {π_(k),{μ_(k) ^((p)),Σ_(k) ^((p))}_(p=1) ^(P)}_(k=1) ^(K) equals the TP-GMM introduced earlier and, for each k, describe the emission probability, i.e., the probability of observation, corresponding to state k. In an HSMM the number of states correspond to the number of Gaussian components in the “attached” TP-GMM. In general, HSMM states are Gaussian distributions, which means that its observation probability distribution is represented as a classical GMM. To render HSMM to be object-centric, the observation probabilities can be parametrized as it is done in TP-GMM to obtain a TP-HSMM.

Given a certain sequence of observed data points

, the associated sequence of states in

is given by s_(t)=s₁, s₂ . . . s_(t). The probability of data point ξ_(t) belonging to state k (i.e., s_(t)=k) is given by the forward variable a_(t)(k)=p(s_(t)=k,{

):

${a_{t}(k)} = {\sum\limits_{\tau - 1}^{t - 1}{\sum\limits_{h - 1}^{K}{{a_{t - \tau}(h)}a_{hk}{N\left( {\left. \tau \middle| \mu_{k}^{D} \right.,\sigma_{k}^{D}} \right)}o_{\tau}^{t}}}}$ where o_(τ) ^(t)=

N(

) is the emission probability, where ({circumflex over (μ)}_(l,k),{circumflex over (Σ)}_(l,k)) are derived from ({circumflex over (Σ)}_(t,k))⁻¹=Σ_(p=1) ^(P)({circumflex over (Σ)}_(t,k) ^((p)))⁻¹,{circumflex over (μ)}_(t,k)={circumflex over (Σ)}_(t,k)Σ_(p=1) ^(P)({circumflex over (Σ)}_(t,k) ^((p)))⁻¹{circumflex over (μ)}_(t,k) ^((p)) as shown above. Furthermore, the same forward variable can also be used during reproduction to predict future steps until T_(m). In this case however, since future observations are not available, only transition and duration information are used, i.e., by setting N(

)=1 for all k and

>t in a_(t)(k)=Σ_(τ-1) ^(t-1)Σ_(h-1) ^(K)a_(t-τ)(h)a_(hk)N(τ|μ_(k) ^(D),σ_(k) ^(D))o_(τ) ^(t). At last, the sequence of the most-likely states s*_(T) _(m) =s*₁, s*₂ . . . s*_(T) _(m) is determined by choosing s*_(t)=argmax_(k)a_(t)(k),∀1≤t≤T_(m).

All demonstrations are recorded from multiple frames. Normally, these frames are closely attached to the objects in O_(a) _(h) . For example, the skill “insert the peg in the cylinder” involves the objects “peg” and “cylinder”, and the associated demonstrations are recorded from both the robot, the “peg” and the “cylinder” frames, respectively.

In addition, consider a set of pre-defined predicates, denoted by B={b_(i), b₂, . . . , b_(L)}, representing possible geometric relations among the objects of interest. Here, predicates b∈B are abstracted as Boolean functions taking as inputs the status of several objects while outputting whether the associated geometric relation holds or not. For instance, grasp:=O→B indicates whether an object is grasped by the robot arm; within: O×O→B indicates whether an object is inside another object; and onTop: O×O→B indicates whether an object is on the top of another object. Note that these predicates are not bound to specific manipulation skills but rather shared among them. Usually, such predicate functions can be easily validated for the robot arm states and the object states (e.g., position and orientations).

Finally, a goal specification G is given as a propositional logic expression over the predicates B, i.e., via nested conjunction, disjunction and negation operators. In general, the goal specification G represents the desired configuration of the arm and the objects, assumed to be feasible. As an example, one specification could be “within(peg,cylinder)ΛonTop(cylinder, box)”, i.e., “the peg should be inside the cylinder and the cylinder should be on top of the box”.

In the following, a problem can be defined as follows: Given a set of demonstrations D for skills A and the goal G, the objective is

-   a) to learn a TP-HSMM model M_(θ) of the form     M _(θ) ={{a _(kh)}_(h=1) ^(K),(μ_(k) ^(D),σ_(k) ^(D)),π_(k),{μ_(k)     ^((p)),Σ_(k) ^((p))}_(p=1) ^(P)}_(k=1) ^(K),     for each demonstrated skill a_(h) and reproduce the skill for a     given final configuration G. For the reproduction of the skill the     sequence of states obtained through Viterbi algorithm can be used. -   b) to construct a PDDL model P of the form     P=(P _(D) ,P _(P))     where Objects, Initial State and Goal Specification define a problem     instance P_(P), while Predicates and Actions define the domain of a     problem P_(D); -   c) to derive and subsequently execute the sequence of manipulation     skills, such that the probability of achieving the given goal     specification G is maximized. Once the domain and problem files are     specified, a PDDL (Planning Domain Definition Language.) planner has     to find a sequence of actions to fulfill the given goal     specification, starting from the initial state.

The PDDL model P includes a domain for the demonstrated skills and a problem instance given the goal specification G.

The Planning Domain Definition Language (PDDL) is the standard classic planning language. Formally, the language consists of the following key ingredients:

-   -   Objects, everything of interest in the world;     -   Predicates, object properties and relations;     -   Initial States, set of grounded predicates as the initial         states;     -   A goal Specification, the goal states; and     -   Actions, how predicates are changed by an action and also the         preconditions on the actions.

In the example embodiment described herein, motion planning is performed at the end-effector/gripper trajectory level. This means it is assumed that a low-level motion controller is used to track the desired trajectory.

The example method for planning a manipulation task is described in detail with respect to the flowchart of FIG. 2.

Firstly, a TP-HSMM model M_(θ) is to be learned for each demonstrated skill a_(h) and reproduce the skill for a given final configuration G.

One demonstrated skill a_(h)∈A is considered. As described above, the set of available demonstrations, recorded in P frames, is given by D_(a) _(h) ={ξ_(t)}_(t=1) ^(N). Furthermore, the set of objects involved with skill a_(h) is given by O_(a) _(h) . The P frames are directly attached to the objects in O_(a) _(h) .

Given a properly chosen number of components K which correspond to the TP-HSMM states, which are the Gaussian components representing the observation probability distributions, the TP-HSMM model M_(a) _(h) abstracting the spatio-temporal features of trajectories related to skill a_(h), can be learned in step S1 using, e.g., an EM-like algorithm. This is beneficial as only one model for the general skill is constructed. FIG. 3 shows an example of an HSMM for “pick the peg” skill that contains 10 demonstrations for “pick from top” and “pick from side”. The learned HSMM model in the global frame has a single initial HSMM state from which two branches encode the two different instantiations of the same “pick” skill. FIG. 3 shows illustrations for HSMM states of a learned skill wherein demonstration trajectories in 2D(left) and transition probabilities between associated states are shown.

A final goal configuration G is provided in step S2 which can be translated into the final state of the end effector 22 x_(G)∈

³×S³×

¹. This configuration can be imposed as the desired final observation of the reproduction, i.e., ξ_(T) _(m) ≡x_(G). Similarly, the initial configuration of the end-effector 22 can be imposed as the initial observation, i.e., ξ₀≡x₀. Following, the most likely sequence s*_(T) _(m) given only by ξ₀ and ξ_(T) _(m) .

The forward variable of formula

${a_{t}(k)} = {\sum\limits_{\tau - 1}^{t - 1}{\sum\limits_{h - 1}^{K}{{a_{t - \tau}(h)}a_{hk}{N\left( {\left. \tau \middle| \mu_{k}^{D} \right.,\sigma_{k}^{D}} \right)}o_{\tau}^{t}}}}$ allows to compute the sequence of marginally most probable states, while we are looking for the jointly most probable sequence of states given the last observation ξ_(T) _(m) . As a result, when using above formula there is no guarantee that the returned sequence s*_(T) _(m) will match both the spatio-temporal patterns of the demonstrations and the final observation. In terms of the example in FIG. 3, it may return the lower branch as the most likely sequence (i.e., to grasp the object from the side), even if the desired final configuration is that the end effector 22 is on the top of object.

To overcome this issue, in step S3 a modification of the Viterbi algorithm is used. Whereas the classical Viterbi algorithm has been extensively used to find the most likely sequence of states (also called the Viterbi path) in classical HMMs that result in a given sequence of observed events, the modified implementation differs in that: (a) it works on HSMM instead of HMM; and that (b) most observations except the first and the last ones are missing.

Specifically, in the absence of observations the Viterbi algorithm becomes

${{\delta_{t}(j)} = {\max\limits_{d \in D}{\max\limits_{i \neq j}{{v_{t - d}(i)}a_{ij}{p_{j}(d)}{\prod\limits_{t^{\prime} = {t - d + 1}}^{t}{{\overset{\sim}{b}}_{j}\left( \xi_{t^{\prime}} \right)}}}}}}{{\delta_{1}(j)} = {{b_{j}\left( o_{1} \right)}\pi_{j}{p_{j}(1)}}}{where}{{{\overset{\sim}{b}}_{j}\left( \xi_{t^{\prime}} \right)} = \left\{ \begin{matrix} {{N\left( {\left. \xi_{t^{\prime}} \middle| {\hat{\mu}}_{j} \right.,\underset{j}{\hat{\sum}}} \right)},} & {t = {{1 \vee t} = T}} \\ {1,} & {1 < t < T} \end{matrix} \right.}$

The Viterbi algorithm is modified to include missing observations, which is basically what is described for variable ‘b’. Moreover, the inclusion of duration probabilities p_(j)(d) in the computation of variable ‘d_t(j)’ makes it work for HSMM.

At each time t and for each state j, the two arguments that maximize equation δ_(t)(j) are recorded, and a simple backtracking procedure can then be used to find the most probable state sequence s*_(T) _(m) .

The above modified Viterbi algorithm provides the most likely state sequence for a single TP-HSMM model that produces the final observation δ_(T). As multiple skills are used, these models need to be sequenced and δ_(t)(j) has to be computed for each individual model M_(a) _(h) . However, only ξ₁ and ξ_(T) can be observed, thus some models will not produce observations. An additional challenge emerges when sequencing HSMMs: as the state transition probabilities between subsequent HSMM states are unknown, the Viterbi algorithm cannot be applied directly to find the optimal state sequence.

As a next step, symbolic abstractions of the demonstrated skills allow the robot to understand the meaning of each skill on a symbolic level, instead of the data level of HSMM. This may generalize a demonstrated and learned skill. Hence, the high-level reasoning of the herein described PDDL planner, needs to understand how a skill can be incorporated into an action sequence in order to achieve a desired goal specification starting from an initial state. A PDDL model contains the problem instance P_(p) and the domain P_(D).

While the problem P_(p) can be easily specified given the objects O, the initial state and the goal specification G, the key ingredient for symbolic abstraction is to construct the actions description in the domain P_(D) for each demonstrated skill, wherein P_(D) should be invariant to different task parameters.

a_(h)∈A is a symbolic representation for one demonstrated skill in PDDL form. The learned TP-HSMM M_(a) _(h) contains a task-parametrized model {π_(k),{μ_(k) ^((p)),Σ_(k) ^((p))}_(p=1) ^(P)}_(k=1) ^(K).

For each model, it is then possible to identify two sets, each of which containing initial and final states and denoted by

,

⊆{1, . . . , K}, respectively.

To construct the preconditions of a skill, the segments of demonstrations that belong to any of the initial state are to be identified, and to further derive the low-variance geometric relations which can be mapped into the set of predicates B. For each initial state i∈

, its corresponding component in frame p is given by (μ_(i) ^(p),Σ_(i) ^((p))) for p=1, . . . , P. These frames correspond to objects {o₁, . . . , o_(P)}, i.e., skill a_(h) interacts with these objects. For each demonstration {ξ_(t) ^((p))}_(t=1) ^(T) ^(m) ∈D_(a) _(h) , the most-likely sequence s*_(T) _(m) =s*₁s*₂ . . . s*_(T) _(m) can be computed as described above. If the first T_(i) time steps correspond to state i, namely, s*₁=s*₂= . . . =s*_(T) _(i) =i, one instance of the predicate b_(l)∈B can be evaluated at state i along this segment: b _(l)(o ₁ , . . . ,o _(P))=T, if

∑ t = 1 T i b ℓ ( o 1 t , … , o P t ) = T T i > η where 0<η<1 is a design parameter (probability threshold). o₁ ^(t), . . . , o_(P) ^(t) are object states computed based on the recorded frame coordinates {b_(t) ^((P)),A_(t) ^((p))}_(p=1) ^(P) and object geometric dimensions. Denote by B_(i) the set of instantiated predicates that are True within state i,∀i∈

. As a result, the overall precondition of skill a_(h) is given by the disjunction of the conjunction of these predicates for each initial state, i.e.,

${{PreCond}\left( a_{h} \right)} = {\underset{i \in \mathcal{J}}{\vee}{\underset{b \in B_{i}}{\land}b}}$ where ∇ and Λ are the disjunction and conjunction operations. Similarly, to construct the effect of a skill, the procedure described above can be applied to the set of final states

. In particular, for each final state f∈

, the set of instantiated predicates that are True within f is denoted by B_(f). However, in contrast to the precondition, the effect cannot contain a disjunction of predicates. Consequently, the effect of skill a_(h) is given by

${{Effect}\left( a_{h} \right)} = {\underset{f \in \mathcal{F}}{\land}\underset{b \in B_{i}}{\land}b}$ as the invariant predicates common for all of the final states. Based on the above elements, the PDDL model P can be generated in an automated way. More importantly, the domain P_(D) can be constructed incrementally whenever a new skill is demonstrated and its descriptions are abstracted as above. On the other hand, the problem instance P_(p) needs to be re-constructed whenever a new initial state or goal specification is given.

Following, it is referred to the planning and sequencing of trained and abstracted skills. The PDDL definition P has been constructed, which can be directly fed into any compatible PDDL planner. Different optimization techniques can be enforced during the planning, e.g., minimizing the total length of the plan or total cost. Denote by a*_(D)=a*₁=a*₂ . . . a*_(D) the generated optimal sequence of skills, where a*_(d)∈A holds for each skill. Moreover, denote by M_(a*) _(d) the learned TP-HSMM associated with a*_(d).

Given this sequence a*_(D), each skill within a*_(D), is reproduced as the end-effector trajectory level, so to maximize the probability of satisfying the given goal G.

The learned TP-HSMM encapsulates a general skill that might have several plausible paths and the choice relies heavily on the desired initial and final configurations. To avoid incompatible transitions from one skill to the next, a compatibility measure shall be embedded while concatenating the skills within a*_(D). Particularly, the proposed solution contains three main steps:

-   a) Cascade the TP-HSMMs of each skill within a*_(D) into one     complete model {circumflex over (M)}_(a*) _(D) , by creating     transition probabilities according to the divergence between the     final states/components of the previous manipulation skill and the     initial states of the current manipulation skill.

Since the transition from one skill to another is never demonstrated, such transition probabilities are computed from the divergence of emission probabilities between the sets of final and starting states. Particularly, consider two consecutive skills a_(d)* and a_(d+1)* in α_(D)* . The transition probability from one final state f of M_(a) _(d) * to one starting state i of M_(a) _(d+1) * is given by

$\left. \left. {{{a_{fd} \propto {\exp\left( {{- \alpha}*{\sum\limits_{p \in P_{c}}^{}{{KL}\left( {N\left( \left( {\mu_{f}^{(p)},\sum\limits_{f}^{(p)}} \right) \right.} \right.}}} \right.}}}{N\left( {\mu_{i}^{(p)},\sum\limits_{i}^{(p)}} \right)}} \right) \right)$ where KL(⋅∥⋅) is a KL-divergence (Kullback-Leibler-Divergence; see also Kullback, S.; Leibler, R. A., “On Information and Sufficiency.” Ann. Math. Statist. 22 (1951), no. 1, 79-86. doi:10.1214/aoms/1177729694, P_(c) is the set of common frames between these two skills, and a≥0 is a design parameter. The outgoing probability of any final state should be normalized. This process is repeated for all pairs of starting and final states between consecutive skills in a_(D)* . In this way, one complete model {circumflex over (M)}_(a) _(D) _(*) has been created for the desired sequence of skills.

-   b) Find the most-likely complete state sequence ŝ*_(T) _(D) within     {circumflex over (M)}*_(a) _(D) given the initial and goal     configurations. Given the derived complete TP-HSMM {circumflex over     (M)}*_(T) _(D) , the observed initial state and the goal     specification, the above modified Viterbi algorithm can be applied     to find the most-likely state sequence ŝ*_(T) _(D) . -   c) Generate the robot end-effector trajectory that optimally tracks     ŝ*_(T) _(D) , namely, to reproduce all skills in a*_(D). Given the     states sequence ŝ*_(T) _(D) , a linear quadratic regulator (LQR) can     be applied to generate the necessary control commands to reproduce     the optimal state sequence ŝ*_(T) _(D) . A step-wise reference is     obtained from {circumflex over (μ)}_(s) _(t) ,∀s_(t)∈ŝ*_(T) _(D) ,     which is tracked by the LQR¹ using associated tracking precision     matrices {circumflex over (Σ)}_(s) _(t) ⁻¹. As the robot state x     lies in the Riemannian manifold M_(R)=     ₃×S³×     ¹, the emission probabilities are therefore Riemannian Gaussian, and     thus the step-wise reference is also given in M_(R). This implies     that the required linear system dynamics for the end-effector 22     cannot be defined on M_(R), since it is not a vector space. However,     the linear tangent spaces can be exploited to achieve a similar     result. Specifically, the state error between the desired reference     {circumflex over (μ)}_(s) _(t) and current robot state x_(t) can be     computed using the logarithmic map

log_(μ̂_(s_(t)))(x_(t)) that projects the minimum length path between {circumflex over (μ)}_(s) _(t) and x_(t) into the Euclidean space

μ ^ s t M R . The covariance matrices {circumflex over (Σ)}_(s) _(t) describe the variance and correlation of the robot state variables in a tangent space

μ ^ s t M R . Under the above assumptions, the control objective in the tangent space can be formulated as

${c(u)} = {\int{\left( {{{\log_{{\hat{\mu}}_{s_{t}}}\left( x_{t} \right)}^{T}{{\hat{\sum}}_{s_{t}}^{- 1}{\log_{{\hat{\mu}}_{s_{t}}}\left( x_{t} \right)}}} + {u_{t}^{T}{Ru}_{t}}} \right){dt}}}$

-    where u is the control input and R regulates the control effort. To     retrieve only the reference trajectory, it can be assumed double     integrator dynamics, which provides smooth transitions between     consecutive {circumflex over (μ)}_(s) _(t) .

Finally, in step S4 the concatenated sequence of manipulation skills is executed. 

What is claimed is:
 1. A computer-implemented method for planning a manipulation task of a robot, comprising the following steps: learning a number of manipulation skills and generating a symbolic abstraction of each of the manipulation skills, by which a respective statistic model of a transition to a respective final state from a respective current state is determined for each of the learned manipulation skills; determining a concatenated sequence of a subset of the manipulation skills selected from the number of learned manipulation skills based on their symbolic abstractions so that a given goal specification indicating a given complex manipulation task is satisfied, wherein: the determining of the concatenated sequence includes generating a complete model corresponding to a combination, in the sequence, of all of the subset of the manipulation skills; and the generating of the complete model is performed by computing, for each of a plurality of pairwise combinations of individual manipulation skills of the subset, a respective probability of a transition from a first one of the individual manipulation skills of the respective pairwise combination to a second one of the individual manipulation skills of the respective pairwise combination according to a divergence of emission probabilities between an end state of the first one of the individual manipulation skills of the respective pairwise combination and an initial state of the second one of the individual manipulation skills of the respective pairwise combination; and executing the sequence of manipulation skills.
 2. The method according to claim 1, wherein: the statistical model is a Task Parameterized Hidden Semi-Markov model (TP-HSMM); and the learning of the number of manipulation skills includes, for each of the number of manipulation skills: demonstrating and recording a plurality of manipulation trajectories; determining a respective instance of the TP-HSMM depending on the plurality of manipulation trajectories of the respective manipulation skill; and determining a respective one of the symbolic abstractions.
 3. The method according to claim 2, wherein the generating of the symbolic abstractions of the manipulation skills includes constructing a Planning Domain Definition Language (PDDL) model in which objects, the initial states, and a goal specification define problem instances, and in which predicates and actions define a domain of a given manipulation, wherein the symbolic abstractions of the manipulation skills use a classical PDDL planning language.
 4. The method according to claim 2, where the determining of the concatenated sequence of manipulation skills is performed such that a probability of achieving the given goal specification is maximized, wherein a PDDL planning step is used to find a sequence of actions to fulfill the given goal specification, starting from a given one of the initial states.
 5. The method according to claim 2, where the transition probabilities are determined using Expectation-Maximization.
 6. The method according to claim 2, wherein the symbolic abstractions are determined by mapping low-variance geometric relations of segments of the manipulation trajectories into a set of predicates.
 7. The method according to claim 2, wherein the determining of the concatenated sequence of manipulation skills includes optimizing a goal of minimization of a total length of a combined trajectory of the concatenated sequence.
 8. The method according to claim 7, wherein the determining of the concatenated sequence of manipulation skills includes selectively reproducing one or more of the manipulation skills of the sequence of manipulation skills so as to maximize a probability of satisfying the given goal specification.
 9. The method according to claim 7, wherein the determining of the concatenated sequence of manipulation skills further includes searching a most-likely complete state sequence between initial and goal states of the complex manipulation task as a whole using a Viterbi algorithm.
 10. The method according to claim 7, wherein the determining of the concatenated sequence of manipulation skills further includes searching a most-likely complete state sequence between initial and goal states of the manipulation task as a whole using a Viterbi algorithm that includes missing observations and duration probabilities.
 11. The method according to claim 1, wherein an individual one of the manipulation skills of the subset is included multiple times within the concatenated sequence.
 12. A device for planning a manipulation task of a robot, the device comprising a processor, the processor configured to: learn a number of manipulation skills, thereby generating a symbolic abstraction of each of the manipulation skills and by which a respective statistic model of a transition to a respective final state from a respective current state is determined for each of the learned manipulation skills; determine a concatenated sequence of a subset of the manipulation skills selected from the number of learned manipulation skills based on their symbolic abstractions so that a given goal specification indicating a complex manipulation task is satisfied; and instruct execution of the sequence of manipulation skills; wherein: the determination of the concatenated sequence includes generating a complete model corresponding to a combination, in the sequence, of all of the subset of the manipulation skills; and the generating of the complete model is performed by computing, for each of a plurality of pairwise combinations of individual manipulation skills of the subset, a respective probability of a transition from a first one of the individual manipulation skills of the respective pairwise combination to a second one of the individual manipulation skills of the respective pairwise combination according to a divergence of emission probabilities between an end state of the first one of the individual manipulation skills of the respective pairwise combination and an initial state of the second one of the individual manipulation skills of the respective pairwise combination.
 13. A non-transitory machine-readable storage medium on which is stored a computer program that is executable by a computer and that, when executed by the computer, causes the computer to perform a method for planning a manipulation task of a robot, the method comprising the following steps: learning a number of manipulation skills and generating a symbolic abstraction of each of the manipulation skills, by which a respective statistic model of a transition to a respective final state from a respective current state is determined for each of the learned manipulation skills; determining a concatenated sequence of a subset of the manipulation skills selected from the number of learned manipulation skills based on their symbolic abstractions so that a given goal specification indicating a given complex manipulation task is satisfied, wherein: the determining of the concatenated sequence includes generating a complete model corresponding to a combination, in the sequence, of all of the subset of the manipulation skills; and the generating of the complete model is performed by computing, for each of a plurality of pairwise combinations of individual manipulation skills of the subset, a respective probability of a transition from a first one of the individual manipulation skills of the respective pairwise combination to a second one of the individual manipulation skills of the respective pairwise combination according to a divergence of emission probabilities between an end state of the first one of the individual manipulation skills of the respective pairwise combination and an initial state of the second one of the individual manipulation skills of the respective pairwise combination; and executing the sequence of manipulation skills. 